MLLM-DataEngine: An Iterative Refinement Approach for MLLM

Despite the great advance of Multimodal Large Language Models (MLLMs) in both instruction dataset building and benchmarking, the independence of training and evaluation makes current MLLMs hard to further improve their capability under the guidance of evaluation results with a relatively low human cost. In this paper, we propose MLLM-DataEngine, a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop iteration, the MLLM-DataEngine first analyze the weakness of the model based on the evaluation results, then generate a proper incremental dataset for the next training iteration and enhance the model capability iteratively. Compared with previous data collection methods which are separate from the benchmarking, the data generated by MLLM-DataEngine shows better targeting, quality, and correctness. For targeting, we propose an Adaptive Bad-case Sampling module, which adjusts the ratio of different types of data within each incremental dataset based on the benchmarking results. For quality, we resort to GPT-4 to generate high-quality data with each given data type. For correctness, prompt design is critical for the data generation results. Rather than previous hand-crafted prompt, we propose an Interactive Prompt Optimization strategy, which optimizes the prompt with the multi-round interaction between human and GPT, and improve the correctness of generated data greatly. Through extensive experiments, we find our MLLM-DataEngine could boost the MLLM capability in a targeted and automatic manner, with only a few human participation. We hope it could be a general solution for the following MLLMs building. The MLLM-DataEngine has been open-sourced and is now available at https://github.com/opendatalab/MLLM-DataEngine.Comment: Code and models are available at https://github.com/opendatalab/MLLM-DataEngin

Mapping Impervious Surface Using Phenology-Integrated and Fisher Transformed Linear Spectral Mixture Analysis

The impervious surface area (ISA) is a key indicator of urbanization, which brings out serious adverse environmental and ecological consequences. The ISA is often estimated from remotely sensed data via spectral mixture analysis (SMA). However, accurate extraction of ISA using SMA is compromised by two major factors, endmember spectral variability and plant phenology. This study developed a novel approach that incorporates phenology with Fisher transformation into a conventional linear spectral mixture analysis (PF-LSMA) to address these challenges. Four endmembers, high albedo, low albedo, evergreen vegetation, and seasonally exposed soil (H-L-EV-SS) were identified for PF-LSMA, considering the phenological characteristic of Shanghai. Our study demonstrated that the PF-LSMA effectively reduced the within-endmember spectral signature variation and accounted for the endmember phenology effects, and thus well-discriminated impervious surface from seasonally exposed soil, enhancing the accuracy of ISA extraction. The ISA fraction map produced by PF-LSMA (RMSE = 0.1112) outperforms the single-date image Fisher transformed unmixing method (F-LSMA) (RMSE = 0.1327) and the other existing major global ISA products. The PF-LSMA was implemented on the Google Earth Engine platform and thus can be easily adapted to extract ISA in other places with similar climate conditions.Peer Reviewe

Urban Treetop Detection and Tree-Height Estimation from Unmanned-Aerial-Vehicle Images

Individual tree detection for urban forests in subtropical environments remains a great challenge due to the various types of forest structures, high canopy closures, and the mixture of evergreen and deciduous broadleaved trees. Existing treetop detection methods based on the canopy-height model (CHM) from UAV images cannot resolve commission errors in heterogeneous urban forests with multiple trunks or strong lateral branches. In this study, we improved the traditional local-maximum (LM) algorithm using a dual Gaussian filter, variable window size, and local normalized correlation coefficient (NCC). Specifically, we adapted a crown model of maximum/minimum tree-crown radii and an angle strategy to detect treetops. We then removed and merged the pending tree vertices. Our results showed that our improved LM algorithm had an average user accuracy (UA) of 87.3% (SD± 4.6), an average producer accuracy (PA) of 82.8% (SD± 4.1), and an overall accuracy of 93.3% (SD± 3.9) for sample plots with canopy closures less than 0.5. As for the sample plots with canopy closures from 0.5 to 1, the accuracies were 78.6% (SD± 31.5), 73.8% (SD± 10.3), and 68.1% (SD± 12.7), respectively. The tree-height estimation accuracy reached more than 0.96, with an average RMSE of 0.61 m. Our results show that the UAV-image-derived CHM can be used to accurately detect individual trees in mixed forests in subtropical cities like Shanghai, China, to provide vital tree-structure parameters for precise and sustainable forest management.National Key R&D Program of ChinaNational Natural Science Foundation of ChinaChina Postdoctoral Science FoundationPeer Reviewe

Spatiotemporal Evolution of Urban Agglomeration and Its Impact on Landscape Patterns in the Pearl River Delta, China

An urban agglomeration is the engine of regional and national economic growth, but also causes many ecological and environmental issues that emerge from massive land changes. In this study, the spatiotemporal evolution of an urban agglomeration was quantified and its impacts on the urban and regional landscape patterns were evaluated. It showed that the urbanized land area of the Pearl River Delta Urban Agglomeration (PRDUA) in China nearly quadrupled, having linearly increased from 1819.8 km2 to 7092.2 km2 between 1985 and 2015. The average annual growth rate presented a bimodal wave-like pattern through time, indicating that the PRDUA has witnessed two rounds of the urbanization process. The growth modes (e.g., leapfrog, edge-expansion, infilling) were detected and they exhibited co-existing but alternating dominating patterns during urbanization, demonstrating that the spatiotemporal evolution of the urban development of the PRDUA follows the “spiral diffusion-coalescence” hypothesis. The morphology of the PRDUA presented an alternating dispersal-compact pattern over time. The city-level and regional-level landscape patterns changed synchronously with the spatiotemporal evolution of the PRDUA over time. The urbanization of the PRDUA increased both the complexity and aggregation of the landscape, but also resulted in an increasing fragmentation and decreasing connectivity of the natural landscape in the Pearl River Delta region. These findings are helpful for better understanding how urban agglomerations evolve and in providing insights for regional urban planning and sustainable land management.Natural Science Foundation of ChinaNational Key R&D Program of ChinaChina Postdoctoral Science FoundationJoint-PhD project of Shanghai Jiao Tong University and The University of MelbournePeer Reviewe

Mapping Impervious Surface Using Phenology-Integrated and Fisher Transformed Linear Spectral Mixture Analysis

The impervious surface area (ISA) is a key indicator of urbanization, which brings out serious adverse environmental and ecological consequences. The ISA is often estimated from remotely sensed data via spectral mixture analysis (SMA). However, accurate extraction of ISA using SMA is compromised by two major factors, endmember spectral variability and plant phenology. This study developed a novel approach that incorporates phenology with Fisher transformation into a conventional linear spectral mixture analysis (PF-LSMA) to address these challenges. Four endmembers, high albedo, low albedo, evergreen vegetation, and seasonally exposed soil (H-L-EV-SS) were identified for PF-LSMA, considering the phenological characteristic of Shanghai. Our study demonstrated that the PF-LSMA effectively reduced the within-endmember spectral signature variation and accounted for the endmember phenology effects, and thus well-discriminated impervious surface from seasonally exposed soil, enhancing the accuracy of ISA extraction. The ISA fraction map produced by PF-LSMA (RMSE = 0.1112) outperforms the single-date image Fisher transformed unmixing method (F-LSMA) (RMSE = 0.1327) and the other existing major global ISA products. The PF-LSMA was implemented on the Google Earth Engine platform and thus can be easily adapted to extract ISA in other places with similar climate conditions

Spatiotemporal Evolution of Urban Agglomeration and Its Impact on Landscape Patterns in the Pearl River Delta, China

An urban agglomeration is the engine of regional and national economic growth, but also causes many ecological and environmental issues that emerge from massive land changes. In this study, the spatiotemporal evolution of an urban agglomeration was quantified and its impacts on the urban and regional landscape patterns were evaluated. It showed that the urbanized land area of the Pearl River Delta Urban Agglomeration (PRDUA) in China nearly quadrupled, having linearly increased from 1819.8 km2 to 7092.2 km2 between 1985 and 2015. The average annual growth rate presented a bimodal wave-like pattern through time, indicating that the PRDUA has witnessed two rounds of the urbanization process. The growth modes (e.g., leapfrog, edge-expansion, infilling) were detected and they exhibited co-existing but alternating dominating patterns during urbanization, demonstrating that the spatiotemporal evolution of the urban development of the PRDUA follows the “spiral diffusion-coalescence” hypothesis. The morphology of the PRDUA presented an alternating dispersal-compact pattern over time. The city-level and regional-level landscape patterns changed synchronously with the spatiotemporal evolution of the PRDUA over time. The urbanization of the PRDUA increased both the complexity and aggregation of the landscape, but also resulted in an increasing fragmentation and decreasing connectivity of the natural landscape in the Pearl River Delta region. These findings are helpful for better understanding how urban agglomerations evolve and in providing insights for regional urban planning and sustainable land management

Mapping Impervious Surface Using Phenology-Integrated and Fisher Transformed Linear Spectral Mixture Analysis

The impervious surface area (ISA) is a key indicator of urbanization, which brings out serious adverse environmental and ecological consequences. The ISA is often estimated from remotely sensed data via spectral mixture analysis (SMA). However, accurate extraction of ISA using SMA is compromised by two major factors, endmember spectral variability and plant phenology. This study developed a novel approach that incorporates phenology with Fisher transformation into a conventional linear spectral mixture analysis (PF-LSMA) to address these challenges. Four endmembers, high albedo, low albedo, evergreen vegetation, and seasonally exposed soil (H-L-EV-SS) were identified for PF-LSMA, considering the phenological characteristic of Shanghai. Our study demonstrated that the PF-LSMA effectively reduced the within-endmember spectral signature variation and accounted for the endmember phenology effects, and thus well-discriminated impervious surface from seasonally exposed soil, enhancing the accuracy of ISA extraction. The ISA fraction map produced by PF-LSMA (RMSE = 0.1112) outperforms the single-date image Fisher transformed unmixing method (F-LSMA) (RMSE = 0.1327) and the other existing major global ISA products. The PF-LSMA was implemented on the Google Earth Engine platform and thus can be easily adapted to extract ISA in other places with similar climate conditions

Circulating cytokines and alcoholic liver disease: a two-sample bidirectional Mendelian randomization study

Increased inflammation in the liver during ethanol exposure is a major feature of alcoholic liver disease (ALD). An important contributing component to the development of ALD is the inflammatory response brought on by immunological response, however the connection between individual circulating cytokines and ALD is still unclear. To ascertain the causation, we conducted a two-sample bidirectional Mendelian randomization research. We extracted 41 cytokines and growth factors of 8293 Europeans and ALD cases of the same ethnicity (1416 cases and 217,376 controls) from the Genome-Wide Association Studies (GWAS) database for two-sample bidirectional MR analysis. Our analyses suggest that higher interleukin-7 (IL-7) levels are associated with an increased risk of ALD (p = 0.028, OR = 1.191,95% CI = 1.019–1.392), while tumor necrosis factor related apoptosis inducing ligand (TRAIL) is a protective factor for ALD (p = 0.032, OR = 0.863, 95% CI = 0.754–0.988) which can reduce the risk of disease occurrence. In addition, genetically predicted ALD does not affect the expression of circulating cytokines regulators. Our study supports that cytokines play a pivotal role in the pathogenesis of ALD. To determine the mechanisms and pathways of action of these biomarkers, further basic research is required to ensure their clinical suitability for preventing and treating ALD.</p